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ABSTRACT 

Case studies of attempts to implement accountability 
and assessment programs at Northeast Missouri State University and 
the University of Missouri, Columbia, illustrate the benefits and 
impacts of such programs. The programs variously incorporated 
demographic data, national standardized test results, and surveys of 
student attitudes. The American College Test, graduate record 
examinations, professional certification instruments, employers' 
surveys, writing and critical thinking instruments, and the 
Cooperative Institutional Program survey scores are used to gather 
quantitative and qualitative information about learning outcomes. 
Political processes and faculty reactions involved in the 
implementation of the programs prove that such facets are central to 
any attempt to establish such a ptc>gram. (TJH) 



********************************************* *********^ 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 
*********************************************************************** 



ERLC 



O 

CO ACCOUNTABILITY AND ASSESSMENT IN AMERICAN HIGHER EDUCATION: TWO CASE STUDIES 

Christopher J. Lucas* 



Introduction 

American higher education has enjoyed unprecedented popular supnort In 
recent years, evidenced In part by what amounts to a doubling of public and 
private support over the past two decades. Likewise, participation has 
reached an all-tliae proportional high, with current enrollment exceeding 18 
million, or roughly half of all U.S. high-school graduates. Public funding 
has been Inspired, historically, by a faith that higher learning affords the 
basis of national well-being, economic progress, and personal 
self-development. Even today post-secondary education is still widely 
perceived as the gateway to the American dream. ^ 

Nevertheless, academe in the late 1980' s confronts a crisis of major 
proportions-and it is one of confidence, or, more precisely, of doubt about 
the ability of higher education to deliver on its promise. The continued 
health and vitality of colleges and universities, many argue, depends upoP 
adherence to rigorous standards of achlevement-f or students, faculty and 
institutions themselves. Recent evidence, however, documented in an array of 
repot us highly critical of undergraduate education, suggests this has not been 
the case. As the 1985 report of the Southern Regional Education Board warned 
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bluntly, "The quality and meaning of undergraduate education has fallen to a 

point at which mere access has lost auch of Its value, 

Signs of scholastic underachlevement , coupled with resulting public 

dissatisfaction, are not lacking. Student performance on professional 

licensure tests and parts of the Graduate Record Examination, for example, 

reportedly has declined markedly In the last decade.^ Faculty complaints 

about students' lessened Interest In learning are as vociferous as they are 
4 

commonplace. State policy makers across the country voice fears that the 
quality of teaching In public collegiate Institutions has begun to suffer.^ 
Even popular periodicals are weighing in with accusations of lack of quality 
controls In higher education.^ 

Conjecture over what has gone wrong abounds. Some attribute a seeming 
loss of focus and sense of priorities by some Institutions to the effects of 
overheated expansion. Others claim exaggerated emphasis upon specialized 
research has led to a neglect of baccalaureate-level Instruction.^ Still 
other critics blame a "permissive social milieu" generally for falling 
standards, lowerea expectations, anemic curricula and a host of other llls.^ 
Whatever the specific factors adduced, the pervasive and disquieting suspicion 
Is that undergraduate education today has entered upon a significant decline. 

Laments over academic mediocrity from public figures and professional 
associations alike have, if anything grown more frequent. They range from 
Secretary of Education William Bennett's 1984 critique of tha condition of the 
humanities. To Reclaim A Legacy; to the 1985 report entitled Integrity in the 
College Curriculum put forth by the Association of American Colleges, in which 
an alleged lack cf institutional accountability was termed nothing less than 
"scandalous." More recently, the Southern Regional Education Board's 
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CoBBission for Educational Quality has appealed for the forging of a "new 

covenant" between higher education and Its various constituencies to Improve 

academic quality and more closely monitor students' academic performance. 

Skeptical about the willingness or ability of colleges to clean their own 

house, some state governments have already begun Imposing more stringent 

admission requirements, mandating achievement tests, and revising funding 

formulae to reward institutions already assessing student learning outcomes or 

otherwise endeavoring to provide evidence of accountability.^^ More of the 

same seems likely in future years. "if American higher education is to 

forestall zhe imposition of a state system of examinations," one observer 

warns, "it will have to improve its own forms of quality control...." He 

adds, "If the academy does not strengthen those controls of its own volition, 

it may find goi rnment moving to do so in ways that jeopardize the core of the 
12 

enterprise ." 

In a similar vein, at the College Board's 1985 annual meeting, William 
Bennett prophesied that should higher-education institutions fail to implement 
their own reliable systems for measuring academic achievement, external 
agencies, including state bodies, would do it for them.^-* Echoing the sane 
point, Clifford Adelman, among others, urged university administrators to 
adopt a mode of "protective anticipation," exercising the initiative before 
finding themselves impelled to take action by outside forces.^* 

The Logic of Assessment 

Common to most reform proposals is the demand for systematic an<S precise 
measurement of learning outcomes. Prerequisite to success is some clear 
specification of goals and objectives. Lacking agreement on its purposes, 
assessment proponents argue, there are no explicit standards or criteria by 
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which an Institution can ascertain the degree to which it is succeeding or 
failing in meeting its aims. A college or university thus must decide what it 
is about before it can begin to gauge its effectiveness in fulfilling an 
academic mission. Only after desired outcomes have been delineated or 
operationally defined, the argument continues, is it possible to administer 
instruments to test for actual performance or achievement. Then, based on the 
congruence or discrepancy between intention and result, appropriate expedients 
can be employed for purposes of remediation and improvement. The value of 
assessment 's therefore two-fold: it imposes a need for precision and clarity 
about intended outcomes and, if successful , it indicates whether such aims 
have been achieved.. 

Institutional self-appraisal takes different forms. Virtually all 
evaluation programs or systems, however, employ some combination of 
post-admission diagnosis, attitudinal and opinion surveys, instructional and 
course evaluations, academic achievement tests, and exit examinations or their 
equivalents. The more ambitious undertakings go further in providing multiple 
measures and observers to track students' intellectual and personal growth 
over some extended period of time, up to and beyond graduation. 
Instrumentation typically encompasses both rationally-normed standardized 
tests and "home-grown" supplements or alternatives. 

Whether the focus is upon general learning, non-cognitive change and 
development, or mastery of specialized knowledge in an academic discipline or 
area, the logic of testing holds that results must be empirically 
demonstrable. Failing this, neither good intentions nor lofty rhetoric is 
sufficient to show whether the college or university is succeeding in the 
tasks to which it devotes itself. 
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"Value-added" assessment represents one highly-touted franework for 
fixing the degree to which an academic Institution Is fulfilling Its purposes. 
By now the outlines of the argument advanced on Its behalf are quite familiar. 
It opens irtth a plea for a conceptual relnterpretatlon of collegiate "quality" 
or "excellence." As Alexander W. Astln, President of UCLA's Higher education 
Research Institute, expresses It, "While American higher education has long 
prided Itself on its 'diversity', the fact remains that most of the more than 
three thousan ' Institutions that make up this vast system have come to embrace 
a remarkably narrow conception of 'excellence. '"^^ Essentially, for most 
Institutions, the pursuit of excellence equates with the pursuit of resources: 
money, ever-expanding facilities, highly credenLlaled faculty, and more 
capable students. Those colleges that succeed In amassing a 
disproportionately large share of resources are regarded as excellent; the 
others tend to be regarded, at best, as mediocre. 

A value-added approach, on the other hand, he argues, poseo an 
alternative to defining quality based on some vague consensus of external 
opinion or an Institution's reputation for garnering resources. It Is based 
Instead on the positive Influence an Institution has on students' Intellectual 
and personal development. A college or university begins by Identifying 
outcome goals. It then selects points In time— from entry to 
graduation—where progress Is to be measured. By comparing pre- and 
post-testf, gains are ascertained. The difference In student performance 
presumably reveals the "value added." The focus of attention so far as 
Institutional quality Is concerned Is not upon mere resource acquisition or 
sheer competlveness , but upon how existing resources are used to enhance 
students' learning. 
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Few If any Institutions of higher learning have yet enplaced a 
"value-added" model of assessnent, though sone have begun to move In that 
direction. Among them Is Northeast Missouri State University (NEMSU), a 
residential undergraduate Institution of about 6,500 students, located In 
Kirksvllle, Missouri. It affords one of the best-known examples of a school 
attempting to satisfy the demand for tangible evidence that students, parents, 
publlr officials and tax payers are receiving their money's worth. 

Oocumentlng a "Positive Difference" 

As the decade of the 1960's drew to a close. Northeast Missouri State 
University faced the likelihood of demographic decline within the geographical 
region it served and from which it drew most cf its students. More ominous 
still for a former normal school was the prospect of plummeting enrollments in 
its teacher education program— long the institution's currlcular mainstay. If 
NEMSU was to ourvlve and preserve its viability as an academic institution, 
programmatic diversification seemed imperative. 

Integral to the refashioning of Northeast as a multipurpose university in 
the early 'yOs was a decision to embark upon a comprehensive effort to assess 
student learning outcomes. A "conceptual system" was devised identifying 
critical contact points and data elements to be observed at each point 
throughout th^ duration of each student's stay at Northeast. Three types of 
data were deemed essential. The first was standard demographic information 
such as gender, race or ethnicity, geographic origin, high-school rank, 
entering test scores, major, grade point average, and so on. The second were 
measures of academic achievement according to national standardized test 
results. The third was surveys of students' attltudlual and perceptual 
changes over time. As summarized tor the National Director of the American 
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Association for Higher Education's Assessment Forum, the "key features" or 
parts of NEMSU's student assessment program as It has evolved since the 
mid-1970s Include: 

(1) a "value-added" component which seeks to measure student growth in 
general knowledge between the beginning of the freshman year and the 
end of the sophomore year through successive administration of 
national standardized examinations; 

(2) a "comparative" component which seeks to demonstrate student 
achievement in the major field by means of a nationally standardized 
test, or when not available, a locally-developed examination; 

(3) an "attitudinal" component which seeks to determine student 
self-perceptioiis of growth and evaluations of the university and its 
services by means of a variety of questionnaire surveys administered 
at various points over the student's academic career. 

The ACT (American College Test) Is strongly recommended for all freshmen 
entering Northeast and is administered to any student who is accepted without 
an ACT score. During the freshman year, all new students are administered the 
ACT College Outcomes Measures Project (COMP) objective test. Test scores 
represent what NEMSU official term the "imput potential" of each student and 
provide baseline data for subsequent comparisons. Additionally, all 
first-time freshmen are required to take a locally-developed mathematics 
placement test during suimer orientation as well as a Summer Orientation 
Student Survey (SOSS), the latter of which generates information on students' 
goals, reasons for college choice %nd overall perceptions of the university. 
Further data on student attitudes, demographic characte .istics and educational 
goals are collected frcm a nationally-normed Cooperative Institutional 
Research Program (CIRP) survey administered to all freshmen. A CIRP follow-up 
survey is completed two years later, thereby providing longitidinal and 
comparative data on changes in the student population. 
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Also used is an Institutional Student Survey (ISS), a locally-developed 
survey distributed biennially to all currently enrolled students. The 
resulting data are aggregated by academic majors and disciplines to provide 
internal institutional comparisons of students' academic progress, p'3rsonal 
growth and reported satisfaction with university programs and services. The 
ACT Withdrawing/Non-returning Student Survey is sent to all former students 
who have elected to discontinue their studies at Northeast. 

During the sophomore year, the ACT assessment and COM? are retaken, thus 
allowing measures of growth la the area of general education. Test results 
are aggregated at the acadamic discipline level, thereby facilitating an 
assessment of curricular strengths and weaknesses on a prograa.-by-program 
basis. 

Each graduating senior is required to complete an exam in a major field. 
Where applicable, all students complete the relevant Graduate Record 
Examination (GRE) field tests, or a preprofessional or certification 
instrument, where available, such as the American Institute of Certified 
Public Accountants (AICPA) Level II Achievement Exam, the Undergraduate 
Assessment Program (UAP) area examinations or the National Teacher 
Examination (NTE). Students in those majors lacking a nationally-normed 
examination are tested using a locally-developed equivalent. Also required of 
each senior is a "capstone" experience consisting of participation in a 
seminar, a research project, completion of a thesis, a recital performance or 
some othei comparable activity. 

A further requirement for graduation is completion of a locally-developed 
Graduating Student Questionnaire (GSQ), which includes questions from the SOSS 
and ISS. The results allow NEMSU to track attitudinal changes, students' 



Page 8 



satisfaction v^th their learning experiences, and perceptions of strengths and 
weaknesses In the curriculum. By maintaining longitudinal data and developing 
profiles of changes in student attitudes and perceptions over time. Northeast 
reportedly is better able to monitor the performance and ef fectlveneas of Its 
instructional and support services. 

Two additional follow-up surveys are used. One is an Alumil Survey 
utilized every third year to measure alumni satisfaction with their academic 
preparation. The other is an Employers Survey, distributed once every thre- 
years to almost 600 employers, requesting information on the academic and 
professional proficiency of the NEMSU graduates they have hired. Currently 
the total reported cost of Northeast's testing program runs slightly in excess 
of $56,000 annually. 

With a vastly-expanded student data base, onlined and available 
longitudinal and historically, by total institution, by discipline and major, 
and by individual student, NEMSU representatives are persuaded they can 
document the "value-added" or positive difference the institution makes in 
terms of student development, not to mention the national competitiveness of 
the '.aiversity's graduates. "Northeast can and does measure the positive or 
negative difference in students' development from their freshmen year through 
graduation, and the comparative strength of Northeast's graduates vis-a-vis 
other students in the country," according to one former officiaL^^ Overall, 
what NEMSU has done, he claims, is to combine "a sense of mission among all 
levels of students with an evaluation process that verifies the impact of the 
total university experience of these studerts," demonstrating that in fact the 
institution can influence its students favorably by making "a positive and 
idenifiable contribution to their mental as well as personal development."" 
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Northeast officials further argue that assessment has not only created 
the conditions for change but also helped to give It direction and 
substance. 2^ Armed with more detailed Information than heretofor has been 
common in higher education, administrators and faculty alike are better able 
to improve their own performance, to review programs as needed, and to 
initiate necessary changes in instruction , curricula, and the organization and 
management of support services. 

Early on, for example, it was found that Northeast's sophomores were 
scoring substantially higher on the ACT English test than before or at 
admission. Faculty were inclined to attribute the gain to a required 
competency-based freshman English course. Closer scrutiny of the data, 
however, revealed that students who had entered with high scores were not 
registering subsequent gains. It was discovered that many had substituted 
"advanced placement" or CLEP credit in English for the freshman course and had 
neither been required nor encouraged to take further writing courses. Hence 
the conclusion initially entertained that faculty had been teachi.ig below the 
level of high-ability students was proved false. Subsequently, requirements 
were modified, aimed at improving the magnitude of improvement shown by 
high-scoring as well as lower-achieving students.^* 

Similarly, when it was found that graduating English majors were not 
performing at national standards, careful investigation showed that many had 
substituted less taxing classes for the more demanding core courses. 
Appropriate action followed, eliminating substitutions for essential courses. 
A significant improvement in student performance reportedly followed soon 
thereafter. Much the same pattern was repeated in the psychology program and 
in curricular requirements for a degree in business accounting. Likewise, 
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when surveys of nursing students' attitudes suggested widespread 
dissatisfaction with the quality of their preparatory program, changes were 
Initiated which led to a substantial Improvement in student morale and program 
acceptance. 

Defenders of NEMSU's assessment program are quick to counter detractors' 
criticisms. A common objection Is that educational "quality" Is too 
Intangible and subjective to assessed. The response Is a flat denial. 
What can be taught can be tested for. The real Issue, allegedly. Is the basis 
upon which judgments about quality are made.^^ To critics who worry about 
"coaching" or "teaching to tests," proponents argue as follows: Available 
evidence Indicates that short-term preparation yields inconsequential gains in 
performance on standardized tests. Hence it would be counter-productive for 
an institution to encourage "cramming." Nonspecific test-taking strategies, 
on the other hand, which do contribute to Improved performance, can and should 
be developed among students. As for long-term preparation, which likewise can 
help performance, the distinction between "coaching" and "schooling" allegedly 
becomes so blurred as to obviate the original objection altogether. 

Some skeptics raise questions about the content validity of standardized 
examinations. Supporters of Northeast's testing program tend to argue either 
that curricula should conform to the content test ad for by national 
examinations or, alternative!. , that alternative examinations can be devised. 
Either way, the review process is said to have salutory benefits in promoting 
awareness of what is or is not Included in a given curriculum or program of 
studies. (Also worth noting is that when faculty volunteers found themselves 
scoring at the ninety-ninth percentile on selected national Instruments, many 
of their original misgivings about the appropriateness of the tests dlsolved.) 
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Official reports of NEMSU's assessment program paint a glowing picture of 
an Institution dedicated to value-added excellence. Ur.derstanJably obscured 
in part by the rhetoric are occasional indications of internal 
dis- ^action. How much support he testing program actually enjoys is 
impossible to determine. Conceivably the vast majorl-y of administrators, 
faculty and students wholeheartedly endorses assessment. Nevertheless, there 
are reports— most of them difficult to document—of faculty skeptical of the 
real benefits involved, of student resentment over the time consumed by 
testing, of possible resistance and occasional sabatoge of evaluation 
procedures. 

Whatever the truth of the matter, the overall Impression conveyed is of a 
hard-driving, ambitious academic administration determined to transform 
Northeast into a major academic institution. Already it has begun to shed its 
regional identity, recently having been designated as the state's "official" 
undergraduate university specializing in liberal-arts education. Thanks in no 
small measure to the publicity surrounding its value-added assessment, in 1983 
Northeast was selected by the American Association of State Colleges and 
Universities as a co-recipient of the prestigious G. Theodore Titau Award for 
Innovation and Change in Higher Education. Finally, NEMSU's performance-based 
approach has won support from the state's legislature, promising possible 
benefits in terms of appropriations for program improvements. However 
Incomplete or flawed may be Northeast's application of the value-added 
concept, it has thus proven to be a remarkably astute move politically. The 
lesson is one not likely to be overlooked by other comparable public 
institutions elsewhere. 
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Northeast Missouri State University Is a relatively homogeneous 
Institution, with an enrollment consisting for the most part of traditional 
undergraduate students. Its program offerings are limited In F-ope compared 
with those of larger universities. Whether or not the student assessment 
program Inaugurated at NEMSU could be duplicated In quite the same fashion 
elsewhere therefore remains open to question. The Columbia campus of the 
University of Missouri (UMC), on the other hand, Is the oldest and largest of 
a four-campus system, with a combined undergraduate and graduate enrollment 
exceeding 22,000 students. It features an extensive array of both graduate 
and undergraduate programs. As a larger, more diverse Institution, Its 
response to political pressure for accountability has assumed a quite 
different form. The challenges It faces as It attempts to fashion a 
comprehensive assessment system are rather more formidable and complex than 
those faced by the state's smaller regional Institutions. 

The UMC Experience 

Missouri Governor John Ashcrcft has long championed assessment as a means 

of bringing accountability to American public higher education. Addressing a 

1985 meeting of the national Governors' Association Task Force on College 

Quality, which he chaired, Ashcroft spoke approvingly of the Impetus to raise 

collegiate admission requirements, to devise standardized core curriculum, to 

Impose more stringent student performance standards and to test for learning 

outcomes. "Public colleges and universities," he declared, "are discovering 

that taxpayers, and elected state officials who represent taxpayers, are 

demanding higher performance from students— and from the Institutions that 
27 

educate them." Likewise, In a 1986 document entitled Time for Results; The 

Governors' I99I Report on Education , which Ashcroft helped draft, strong 
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support was forthcoming for a comprehensive system of assessment as a means of 

promoting accountability In the nation's public system of post-secondary 

28 

learning. 

Again In December of 1986, at a state-wide conference on assessment 

co-sponsored by the Governor and Missouri's Coordinating Board for Higher 

Education, Ashcroft urged representatives of the state's public Institutions 

of higher learning to make assessment their top priority. Sfortly thereafter, 

writing In Higher Educ ation and National Affairs , he announced his expectation 

that each of Missouri's public colleges and universities would have "rigorous, 

systematic programs of student assessment. . .In place on each campus In the 

next academic year [I.e., 1987-88]." Programs were to Include "elements of 

naclonally-nonned data;" they were to be "comprehensive" In assessing 

students' acquisition of knowledge and skills; and the results were to be made 
29 

public.^ 

The Governors' mandate was unequivocal and unmistakable. Less than a 
week later. University of Missouri President C. Peter Magrath, speaking before 
a meeting of the University's Board of Curators, Implicitly endorsed 
Ashcroft' s agenda. Including the requirement that an assessment system be 
operational In the 1987-88 academic year.^° The Board's response was approval 
of a five-point policy statement enunciating guidelines for assessment on the 
University's four campuses. 

At this time and in a subsequent document approved several months later, 
the Board outlined what thereafter would serve as official policy.''^ It 
re-afflrmr' a proviso in its revised Long-Range Plan calling for Improvement 
in the University's "ability to assess the extent to which it accomplishes Its 
instructional mission. "^^ Specifically, the Board wanted to know more about 
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student learning outcomes, post-graduation placements and fitudents' 
self-assesamerits of their educational experiences."'^ Further, the Curators 
mandated th^t student-outcome assessment be used to improve the quality of 
instructional programs; that faculty play a "key role in the design and 
implementation of evaluation strategies..."; that assessment procedures be 
"diverse, sophisticated, and multiple"; and, finally, that results be made 
freely available to "students, faculty, and to the interested and concerned 
public. "^^ 

Meanwhile, undargraduate education had been the topic of the April 1986 
general meeting of \JliC faculty. In presenting an overview of undergraduate 
assessment, former Chancellor Barbara S. Uehling noted, "Today there is a wave 
of criticism across the county, and even self-flagellation, regarding higher 
education. A defense of higher education is not necessary to this crowd, but 
it is useful to think about how we justify ourselves to the outside world." 
Uehling wound nj, her remarks by advocating the use of a variety of approaches 
to assess the quality of the University's teaching efforts. She added, "We 
have nothing to do but gain by self-evaluation."^^ She and other 
administration officials repeatedly took pains to emphasize that the 
University's attempts to assess student learning were not solely the result of 
Governor Ashcroft's mandate. In fact they were only part of a much larger 
total quality assurance program begun long before. 

Indicative of this concern was an initiative taken in March by the UMC 
Provost and Faculty Council to create four separate task forces charged with 
reviewin various aspects of the campus' undergraduate programs. They dealt, 
respectively, with the freshman-year experience, the training of teaching 
assistants, extra-classroom learning resources, and basic core competencies. 
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An ancillary charge to the last-mentioned was the devising of policy 
recommendations for assessing student learning outcomes. 

The Task Force on Basic Competencies Issued an Interim report on 
assessment seven months later. In October of 1986. It began by cautioning 
against exclusive reliance upon objective, nationally standardized testing.''^ 
Also Included was an enthusiastic endorsement of an Innovative Intensive 
writing and critical-thinking assessment Instrument which had recently been 
devised by the Campus Writing Program. It Is noteworthy, perhaps, that the 
Task Force made no mention of yet another on-going assessment effort begun 
previously, one Involving the use of several American College Testing Program 
survey Instruments to gauge student satisfaction with the overall quality of 
their undergraduate experience. This had been the outgrowth of a 1984 
on-campus seminar conducted by the National Center for Higher Education 
Management System (NCHEMS), devoted to the topic of identifying and measuring 
student learning. Its eventual outcome was the administration by the UMC 
Office of Student, Personnel and Auxiliary Services of ACT Student Opinion 
Surveys to a random sample of over 2000 students in May of 1987. Soon 
thereafter the ACT Alumni and Withdrawing /Non-re turning Student Surveys were 
likewise administered to a large randomly-selected population sample. 

The Task Force's report notwithstanding, the Board of Curator's action at 
the end of the Fall term, 1986, calling for the emplacement of a comprehensive 
assessment program by 1987-88 appeared to catcti most of the campus unprepared 
and off-guard. As the holiday recess drew near, the UMC Provost hastily 
convened a six-member Steering Committee on Assessment. Its charge was as 
ambitious as it was urgent: to design an overall campus assessment plan and 
to have it ready for implementation by the Fall of 1987. Clearly, its 
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predecessor-body's recommendation to expand the writing and critical-thinking 

project, Included In the final report of the Task Force on Basic Competencies, 

38 

was Insufficient, What the Governor, the University's President and the 
Board of Curators envisioned now was something much broader and 
farther-reaching In scope* 

The Steering Committee lost no time Is setting to work. By mid-January 
of 1987 It was engaged In Intensive discussion with representatives of the 
Educational Testing Service (ETS) of Princeton, New Jersey, seeking ways of 
satisfying the need for a "natlonally-normed" component to the campus 
assessment plan. Privately, ETS officials expressed dismay over the hurried 
timetable Involved In the decision-making process, a situation they considered 
unprecedented. Their misgivings were shared by members of the Steering 
Committee. Unwilling to sanction the use of any existing test, the Committee 
eventually agreed as an alternative to help pilot test a new ETS Academic 
Profile then under development. It would be made available In two formats: a 

one-hour version for use In program evaluations and a three-hour version for 

38 

Individual student evaluations. The question left open for the moment was 
which version was more suitable for UMC's purposes as an experimental test 
site. 

By the end of January the Steering Committee was prepared to recommend a 
tentative four-point campus assessment plan. Besides the Academic Profile 
test for assessing general education. It urged adoption of the writing and 
critical-thinking assessment project as an Integral program element. Thirdly, 
It encouraged the use of the three ACT Student Opinion Surveys while 
recommending that a fourth Instrument, an employer survey, be devised as well. 
Lastly, it suggested that the Provost Inventory all departments on campus as 
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to their current evaluation procedures. Each academic unit would be asked 
also to outline Its future plans for testing seniors In their major and to 
indicate the availability of appropriate examinations for doing so. 

The Steering Committee's four-point plcn was presented and accepted by 
the University's Board of Curators. Throughout the remainder of the Winter 
1987 term, numerous meetings were held and presentations made to publicize the 
plan across the campus. Especially controversial was the proposed field 
testing of the ETS Academic Profile. Critics pointed out no uniform core 
curriculum existed. Moreover, it was observed, many students were failing to 
complete their general education requirements within a two-year time limit, 
thereby rendering the validity of a standardized test highly suspect. Student 
opposition to the use of the three-hour version of the Profile mounted. 
Ultimately, it was decided to pursue an institutional or program-based 
approach to assessment, using the one-hour test version. Left unanswered was 
a question of how coercive an approach would be -quired to ensure student 
compliance with and participation in the planned testing procedure. 

In July of 1987 the Steering Committee submitted a position paper on 
assessing undergraduate student learning outcomes. It opened with a lengthy 
review of previous and on-going assessment efforts, the Governor's call for 
systematic institutional appraisal, the assessment goals of the University 
revised Long-Range Plan, and the Curator's mandate to the campus. The paper 
reiterated the Committee's lack of support for the idea of using the American 
College Testing College Outcome Measures Project (ACT COMP) , denying it could 
serve as a valid and reliable instrument to measure general education outcomes 
at UMC. Likewise, the Committee voiced doubts about any value-added 
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assessment model, such as that employed by Northeast Missouri State 
University: 

Value-added models are suspect in assessing higher education 
where the outcomes are considered more Important than 
examining the processes of learning. Subtractlve 
test-re-test procedures, separated by more than a year 
between tests, virtually ensure that students will do better 
on the second exam than on the first without providing any 
assurances that the gains can be credited to the curriculum. 
It Is likely that at least part of gain could be 

attributed to an Improvement In the students' test taking 
ability, their maturation, or their experience with the world 
outside. . .the university curriculum. 

The paper's latter section dealt with the previously-approved four-point 
assessment plan: (I) evaluation of writing and critical-thinking skills; (2) 
the use of the ETS Academic Profile; (3) departmental assessments of 
proficiency in the major field; and (4) assessment of the total educational 
environment through a series of student, alumni and employer surveys. Once 
again, the Committee underscored its concern that assessment focus on the 
institution and its programs, not on individual students. To test all of the 
University's 17,000 or so undergraduates using standardized Instruments, it 
was said, would run counter to good sampling theory, would be prohibitively 
expensive, and would entail a "logistical nightmare" never before contemplated 
or attempted by any institur.ior of comparable size and range of mlsslon.^^ 

The paper's concluding section struck a decidedly cautionary note: "UMC 
faculty and students alike are keenly aware for the potential for abuse 
Inherent in a program of arbitrary assessment or testing of students outside 
of the traditional course-related framework." It continued, "Some faculty and 
administrators have also voiced concern that students will view assessment as 
a trivial intrusion upon tielr serious pursuit of a degree or as an annoyance 
to be expedited quickly and will thus put forth minimal effort...." The 
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resultant "outcome," the Committee observed, might therefore reflect poorly on 

. ^ 42 
their mentors. 

Arguing against mandating use of the three-hour version of the ETS 
Academic Profile and a proposal to Incorporate test scores as parts of a 
student's permanent record, the Steering Committee put itself on record In 
favor of a less coercive, non-punltlve approach. Moreover, It firmly opposed 
"the use of assessment data for tenure, promotion, and salary adjustment 
purposes" Involving faculty, or. In the case of students, "as a bar to 
graduating, admission to a specific program, or upper division course." 
Finally, the use of test results for drawing cross-departirental , divisional or 
other comparisons was strongly opposed. ^'^ 

At the opening of the Fall, 1987, the UMC Vice Provost for Instruction 
sent our a leagthy memorandum to deans and departmental chairs outlining a 
"plan of action for assessment" for the academic year.** He reminded 
administrators that assessment of proficiency In the major field would be 
required of all departments and programs offering undergraduate majors. All 
other assessments (surveys and so forth) would be conducted by a sample of 
departments, including all those undergoing program review In 1987-88. 
November was stipulated as the deadline for submission of "a preliminary plan 
outlining the desired goals for graduates In the field, and the ways in which 
program assessment will be conducted with participation of students completing 
&n undergraduate degree in May 1988."*^ 

The Vice Provost indicated that a variety of assessment methodologies 
might be employed, including portfolio reviews, performance reviews, 
laboratory experiences, a capstone exercise, a faculty-developed exit 
examination, natlonally-normed standardized tests, or admission tests taken by 
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pre-professlonal students entering upon a graduate program. To coordinate the 
development of more detailed assessment procedures, he announced, an ad-hoc 
faculty-student committee would be appointed and accorded temporary housing on 
campus By November 18, 1987, virtually all departments Involved had In fact 
submitted detailed plans for assessing the proficiency of their respective 
majors. A follow-up memorandum from the Vice-Provost, dated February 5, 1988, 
laid out a detailed format to be followed by all departments in reporting the 
results of their assessment of student majors. 

Faculty reactions were mixed. Some faculty questioned the cost of 
assessment at a time when other pressing needs remained unmet. There were 
complaints over the apparent lack of appropriate tests for assessment 
purposes. biificulties involved in securing student cooperation troubled 
many. Others remained suspicious of the uses to which assessment data might 
be put. Some anticipated that "teaching to the test" would become a pervasive 
campus syndrome. Most frequently expressed were doubts as to whether any 
assessment process could accurately capture and represent the benefits of the 
total college experience. Others responded that assessment was a political 
reality unlikely to disappear in the foreseeable future, and that realism 
dictated making the best of the situation. Some faculty remained optimistic, 
arguing that assessment should be given a chance and that the , ;ential 
bonellts to be derived were not inconsiderable. 

Matters came to a heid in November of 1987 when a special faculty meeting 
was called in the College of Arts and Science. Proposed for discussion was a 
resolution demanding "through discussion among and approval by the general 
faculty" of any assessment program. An amendment (subsequently defeated) 
urged the College ot decline to participate in any program until such debate 
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and an endorsement had occurred. The meeting was tense. Leading the 
opposition to assessment was Kerby Miller, a UMC professor of history. In 
remarks delivered before the faculty and later reprinted in a local newspaper. 
Miller assailed the prospect of faculty compliance with the proposed program 
as a "political sell out." 

Assessment is fundamentally a teaching issue, he argued, one cutting to 

the very heart of the educational enterprise. Faculty were never adequately 

consulted about the validity of desirability of assessment; they were merely 

called upon to help rationalize and implement it, he claimed. Miller voiced 

apprehension that teaching for understanding, analysis and problem-solving 

would be subordinated to preparation for test-taking. As for the supposed 

success of value-added education at Northeast Missouri State, he alleged that 

class essay test and term papers had been replaced there by machine-graded 

exams designed to familiarize students with the tricks of standardized 

evaluation. "In spite of the pressures exerted upon them," he avowed, "a 

large number of the Kirksville campus's faculty and students regard the 

value-added system as fraudulent and demeaning. A few even silently endeavor 

to sabotage that system, causing administrators no end of trouble identifying 

and discounting intentionally mismarked exams so the scores will not lower the 
47 

campus average." 

Miller questioned the political motives of those proposing assessment: 

It would be pleasant to assume that those purposes and 
motives are benign. But that would be naive, for we ha' ^ few 
reasons to believe that assessment results will be used to 
encourage public recognition of uhe university system's need 
for adeiuate state funding. It is much more likely that high 
bcok^b will be interpreted as evidence that current funding 
levels are adequate, while low scores will be used to justify 
funding ^j)ostponement or even punishment until scores 
improve. 
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He found the argument that the purpose of assessment was to make the 

University prove it was giving the people of Missouri full value for tax 

monies currently expended bitterly ironic. Missouri's per capita funding for 

higher education, he correctly observed, ranks at or near the bottom of even 

the so-called Big Eight schools Inverting the argument, he claimed that low 

assessment scores should demonstrate that Missouri was receiving— and 

deserved— precisely what it paid for. More generous funding, he added, would 

require greater commitment and courage: "It is easier and chtaper to delude 

the public by embracing a superficial 'quick fix' that panders to a low-tax, 

anti-intellectual mentality, puts the university on the defensive and forces 

its campuses to engage in debilitating competition for inadequate 
49 

resources." 

Miller concluded that the University's administrative leadership had 

bowed all too quickly to pressures from external bodies for reasons of 

self-preservation. "Certainly," he commented acidly, "people who remain on 

their knees are less visible targets than those who stand upright." Hence, it 

would fall to the faculty. Miller reiterated, to challenge and refute, rather 

than pander to, what he termed the "chimera" of accountability through 
50 

assessment . 

Following protracted debate, the Arts and Science faculty by a narrow 
vote approved a resolution requiring faculty assent before assessment was 
extended beyond the trial period of 1987-88. Shortly thereafter, the UMC 
Faculty Council announced it planned to appoint a "blue ribbon" committee to 
"assess assessmtnt" ^nd to hold Spring hearings on the issue. This too proved 
controversial. Amidst allegations that the Chair of the Faculty Council 
intended to "pack" the committee with assesbment supporters, debate arose over 
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who would serve as members. Critics were mollified only when self-identified 
critics of UMC's assessment scheme were included on the committee. In a 
separate action at a December 9 meeting, the entire UMC faculty endorsed the 



academic year 1987-88 • 

Field testing of the ETS Academic Profile took place on February 17 and 



Commenting on the lack of cooperation, a UMC associate professor claimed he 
was not surprised. Why should students subject themselves to testing, he 
asked, "merely to satisfy a governor who has not been a friend to high 
education?" He continued: 



More importantly, the governor and company have shown their 
contempt for a democratic educational process by forcing the 
University into compliance with a plan that the faculty and 
students have not even had a chance to discuss. Instead of 
writing edicts for the University, the governor should ask 
more questions, for example how can one improve the quality 
of education in this state without funding it. As more money 
is poured into assessment in another lean fiscal year, less 
is alloted to the areas that need more funds. One can only 
syrnathize with students who see further increases in tuition 
in order to implement the assessment program. 

The same faculty member sardonically congratulated those who had declined to 

participate in the testing program. "Those who chose to play basketball, cliafc 

with friends, or even spend some time in a local watering hole," he affirmed, 

"probably had a more worthwhile educational experience. "^^ 

Thus, as the academic year drev to a close, the institution remained as 

divided as ever over assessment issues. Administrators were pledged to make 

the program work. Departments were engaged in a variety of novel testing 

procedures, albeit some in fairly desultory fashion. Hearings were scheduled 




18, 1988. 



Seme 800 Juniors were asked to volunteeer. Few showed up. 
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by the Faculty Council's assessment committee. Most students were oblivious 
to the commotion or indifferent to it altogether. But faculty critics 
remained as angry and resentful as they had been the the Fall. A majority, 
although a remarkably silent and subdued segme- . of the faculty, appeared 
resigned to more testing. Supporters, insofar as can be judged, remained 
optimistic that the University of Missouri-Columbia had embarked upon a 
worthwhile undertaking offering significant promise for the institution's 
future. On one point, however, all could agree: much experimentation and 
discussion lay ahead. It remained to be seen whether the UMC experience with 
assessment would prove as useful as its enthusiasts hoped for or as 
counter-productive as its detractors anticipated. 

* * * 
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